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1 . Introduction 

The concept of entropy is the fundamental ingredient in information theory and 
statistical mechanics. Given a probability distribution {Pi} i=12L w saus fyi n g 

Pi=h the Boltzmann-Shannon entropy, i.e., the 
ordinary information entropy, is defined by 

w 

S(p 1 ,p 2 ,L ,p w ) = -kY, Pi^Pi, (1) 

where k is a positive constant, which is henceforth set equal to unity for simplicity. This 
quantity is a positive and concave function of {/*,-}. =1 2 L w - It also fulfills the H-theorem. 
Its maximum is realized by the equiprobability distribution, i.e., p t = l/W 
(i = 1, 2, L , W) and the value itself is S = \nW, which is the celebrated Boltzmann 
formula. 

To characterize what the entropy is, Shannon [1] and Khinchin [2] investigated its 
axiomatic foundation. The Shannon-Khinchin axioms are given as follows: 

[ ] S(p 1 , p 2 , L , p w ) is continuous with respect to all its arguments and takes 

its maximum for the equiprobability distribution p t . = 1 / W 
(/ = 1,2,L ,W), 
[ ] S[A,B] = S[A] + S[B\A], 

[ i] S(p v p 2 ,L , p w ,0) = S(p l , p 2 ,L ,p w ). 
In the second axiom, S [A, B] and S [A] are the entropies of the composite system 
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(A, B) with the joint probability distribution p u (A, B) (i = 1, 2, L , W; j = 1, 2, L , t/) 
and the subsystem A with the marginal probability distribution Pi{A) = \", p u (A,B), 
respectively. S^A] stands for the conditional entropy associated with the conditional 
probability distribution p u [B\A) = p ij (A, B) j p t (A). Writing the entropy of p ij {B\A) 
as S [5 1 A i ] , the conditional entropy is given by 

S[B\A] = (s[B\A]) iA) =£ Pi (A) S[B\A]. (2) 

In the special case when A and B are statistically independent, 5 [5 |A] = S[B], leading 
to the additivity: 

S[A,B] = S[A] + S[B]. (3) 

The uniqueness theorem [1,2] states that a quantity satisfying the axioms [ ]-[ ;] is 

uniquely equal to S in eq. (1). 

We note that there is a correspondence relation between the Bayes multiplication law 

and the axiom [ ]: 

Pij {A,B) = Pi (A) Pij (B\A) ^ S[A,B] = S[A] + S[B\A]. (4) 

Now, in the field of statistical mechanics, there is a growing interest in nonextensive 
generalization of Boltzmann-Gibbs theory. In particular, the one initiated by Tsallis [3-5] 
is receiving much attention. This formalism is based on the following single-parameter 
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generalization of the Boltzmann-Shannon entropy: 



j w 

s q ( Pl ,p 2 ,L , Pw ) = — £ (p.y -i 

1 <7 L i=i 



(q>0). 



(5) 



In the limit q— >1, S converges to 5 in eq. (1) (with = 1). This quantity is also 
positive and concave, and satisfies the H-theorem. In addition, as the Boltzmann-Shannon 
entropy, it takes its maximum for the equiprobability distribution p, =l/W 

(i = l, 2, L,W): Its value is S q =(l-q) (W x ~ q -l), which is a monotonically 
increasing function of W. However, the additivity is violated. For statistically 
independent systems, A and B, S q yields 

S q [± B] = S q [A] + S q [B] + (1 - q)S q [A]S q [B] , (6) 

which is referred to as the pseudoadditivity. Clearly, the additivity holds only in the limit 
q — > 1 . Equation (6) has been carefully discussed in Ref. [6] . 

In a recent paper [7], dos Santos has made an interesting discussion about uniqueness 
of the Tsallis entropy in eq. (5). He has shown that that if a quantity is [ t] continuous 

with respect to the probability distribution {p,} (=12L w , [ •] a monotonically increasing 

function of W in the case of the equiprobability distribution p t . = 1 / W (i = 1, 2, L , W) , 
and [ ] satisfies the pseudoadditivity in eq. (6) for the statistically independent systems, 

A and B, and [ 1] fulfills the relation S q (p v p 2 , L , p w ) = S q (p L , p M ) + 

{p L ) q S q {pJ P L ,P 2 I P L ,l^ ,p w /p L ) + {p M )"S q (p 1 /p M ,p 2 /p M ,L ,p w /p M ), where p L = 
^.jj p t and p M = ^._ w +1 , then it is identical to the Tsallis entropy. 
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Here, comparing the set of the Shannon- Khinchin axioms [ ]-[ ;] with that of [ t]- 

[ % we find that the complete parallelism is missing between the two. The main reason 

for this is due to the absence of the concept of "nonextensive conditional entropy". 

In this paper, we present the axioms for the Tsallis entropy by introducing the 
nonextensive conditional entropy. Then we prove the uniqueness theorem for the Tsallis 
entropy. Our proof essentially follows a natural generalization of the line presented in Ref. 
[2]. This improves the discussion given in Ref. [7] and establishes the complete 
parallelism with the Shannon-Khinchin axioms. 



2 . Nonextensive conditional entropy 

In nonextensive statistical mechanics, it is known [4] that the average of a physical 
quantity Q = {<2, 2 L w is given in terms of the normalized ^-expectation value: 



w 

(2> 9 = £ Qi p i = ^ . ( 7 ) 



i=i 



where P f = {p ( ) q / '£._ 1 (/ , ,-) ? ^ s ^ escort distribution associated with p t [8]. To be 
consistent with the nonextensive formalism, we use this concept to generalize the 
definition in eq. (2). For this purpose, we calculate the Tsallis entropy of the conditional 
probability distribution 
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(8) 



From this quantity, we define the nonextensive conditional entropy as follows: 



w 



S q [B\A] = (S q [B\A i ]} =^ . (9) 



Using the definition of the Tsallis entropy, we find that equation (9) can be expressed as 



S \B\A]= qV J (10) 

?L 1 J 1 + (1-^)5, [A] 



An important point here is that, with this definition, a natural nonextensive generalization 
of the correspondence relation in eq. (4) is established in conformity with the 
pseudoadditivity in eq. (6): 

Pij {A, B) = Pi (A) Pij (B\A) 

(ID 

S A A > B \ = S M] + S i[ B \ A ] + d - 9)S q [A]S q [B | A]. 

In fact, the pseudoadditivity in eq. (6) is recovered in the special case when A and B are 
statistically independent each other. 
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3 . Axioms and uniqueness theorem for Tsallis entropy 



Now, the set of the axioms we present for the Tsallis entropy is the following: 

[ ]* S q {Pi> P2' L ' Pw) * s continuous with respect to all its arguments 

and takes its maximum for the equiprobability distribution p t = l/W 
(/ = 1,2,L ,W), 
[ ]* S q [A,B] = S q [A] + S q [B\A] + (l-q)S q [A]S q [B\A], 

[ i]* S q (p v p 2 ,L , p w ,0) = S q (p l , p 2 ,L , p w ). 



Theorem: A quantity satisfying [ ]*-[ ;]* is uniquely equal to the Tsallis entropy. 

Proof: First, let us consider the equiprobability distribution p i = II W (i = 1, 2, L ,W) 
and put 



1 1 1 



L ,— \:=L(W). 



(12) 



From [ ;]*, it follows that 



L q (W) = S q 



ri 1 1 n 

— ,— ,L ,— ,0 

WW w 



( 1 1 



1 1 



yw+i w+i w+i w+i 



) = ^(w+i), 



(13) 



which means that L g (W) is a nondecreasing function of W . 
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Consider m statistically independent systems, A l5 A 2 , L , A m , each of which contains 
r (> 2) equally likely events. Then, we have 



(14) 



Using [ ]* for these independent systems, we find 



S,[A 1 ,A 2 ,L,Aj = £ (l- ? )^[L 9 (r)] 



=^{[i +( i-, )W r4 



(15) 



Since ^[a i; A 2 ,L , A m ] = L ? (r m ), we have 



^i=^{[i+(i-^wr4 



(16) 



Similarly, for any other positive integers n and s equal to or larger than 2, we have 



^") = ^-{[l + (l- ? )L»]"-l}. 



(17) 



It is always possible to take m, r, n, and s which satisfy 



r m <s n < r m+l 



(18) 



Since L is a nondecreasing function, we have the inequalities 



h{r m )<L q {s n )<L q {r m+ % 



(19) 



which lead to 



^{[1 + (1 - q )L q {r)] n - 1} < ^-{[1 + (1 - q)L q {s)\ - 1} 



^{M-^wr 1 - 1 }- 



(20) 



Here, it is necessary to examine two cases, < q < 1 and g > 1, separately. After simple 
algebra, we find that in both cases the following inequalities hold: 



m < In 
n In 



\ + {\-q)L q {s) 



\ + (\-q)L(r) 



^ m 1 

< — + -. 
n n 



(21) 



We note that, in deriving these inequalities, the following Ansatz has to be made: 



<! + (!-<?) L q (r), \ + {\-q)L q {s)<\ for ? >l. 



(22) 



Later, we shall see that this is in fact justified. Also, from eq. (18), it is evident that 



m Ins „ m 1 

— < < — + -. 

n lnr n n 



(23) 



Combining this with eq. (21), we have 



In [l + (l-q)L q (sj 
In 1 + (1- q)L q (r) 



In s 
lnr 



< 



(24) 



Since n can be arbitrarily large, we obtain 



ln[l + (l-q)L q (r)] _\n[l + (l-q)L q ( S )] 



lnr 



In s 



(25) 



where is a separation constant dependent on q . Therefore, we find 



LAr) = 



l-q 



(26) 



Clearly, A(1) = 0. 

Next, let us consider any rational numbers 



(/ = 1,2,L 



(27) 



g ; .The system A is 

assumed to be described by the probability distribution {p t .= g t /g}. =1 2 L ^. We 

construct the system 5 dependent on A as follows. 5 contains g events, which are 
partitioned into W groups: B v , B 2 , L , 5^. fi. (1< j < W) has g j events. Once the 
z'th event A i of the system A was found, i.e., A = A n then, in the system B, g t 
events of the group B j=j have the same conditional probability 1/ g i and all the events of 
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the other groups B M have the vanishing probability. S q of B thus constructed is 
calculated to be 



s X B \ A ]= s q 



( \ i r 



8 i 8 i 8 i 



'J 



(28) 



Therefore, the nonextensive conditional entropy is given by 



S q [B\A] = (S q [B\A] 



(A) 



e (p.ys^A,] 



— !=1 



w 

E (p.) 1 

! = 1 



1-9 



E(".)"fe) A,,l 

J=l 1 



E (p,)' 



(29) 



On the other hand, the composite system (A, B) consists of the events A t B j=i 
(1 < / < W). For a given i, the number of possible events A,. B J=j (1 < z < W) is g ; , and 

8i= 8- Th e 

probability of finding the event A i B j=i is p i x (l / g i ) , which is the equiprobability 1 / g . 
Therefore, S q of the composite system is 



S q [A,B] = L q {g) = ^Ag^-l] 



(30) 



Substituting eqs. (29) and (30) into [ ]* and using eq. (27), we have 
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l-q 

This holds for any rational p i (i = 1, 2, L ,W), but actually for any probability 
distribution {p,}._ 12L w due to the assumption of continuity in [ ]*. 

A remaining task is to determine X(q). For this purpose, it is sufficient to calculate the 
nonextensive conditional entropy using the form in eq. (31) and impose [ ]* on it. 
Consequently, we find 

A(</) = l-</. (32) 

At this stage, we also see that the Ansatz in eq. (22) is in fact justified. Thus, we see that 
S q satisfying [ ]*-[ ;]* is uniquely equal to the Tsallis entropy in eq. (5). (Q.E.D.) 



w 

I {PiY 



q+K(q) 



(31) 



4 . Concluding remarks 

We have constructed the nonextensive conditional entropy in conformity with the 
Bayes multiplication law and the pseudoadditivity of the Tsallis entropy. We have 
generalized the Shannon- Khinchin axioms for the Boltzmann-Gibbs entropy to the 
nonextensive systems. Based on the proposed set of axioms, we have proved the 
uniqueness theorem for the Tsallis entropy. 

Recently, the nonextensive (nonadditive) conditional entropy has been discussed in the 
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quantum context in Ref. [9]. There, it has been shown to give rise to the strongest 
criterion for separability of the density matrix of a bipartite spin- 1/2 system for validity of 
local realism. We also mention that characterization of the Tsallis entropy has been 
considered in Ref. [10] from the viewpoint of the concept of "composability", which 
means that the entropy of the total system composed of statistically independent 
subsystems is expressed as a certain function of the entropies of such subsystems. (The 
additivity of the Boltzmann-Shannon entropy and the pseudoadditivity of the Tsallis 
entropy are the actual examples.) The authors of Ref. [10] has shown that if a quantity 
satisfies the composability (and some other supplementary conditions), then it is given by 
the Tsallis entropy with q>l. 
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